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ABSTRACT 



The objectives of this study were identification and 
verification of key processes that could be used by higher education quality 
administrators to implement continuous improvement programs and benchmarking 
processes. East Tennessee State University’s Continuous Improvement Key 
Process Relationship Matrix was used as the basis of a 44- item questionnaire 
sent to a sample of 49 higher education institutions; 26 replies (53 percent) 
v:ere received. Responses were tabulated for 10 key processes and 34 
associated performance measures. Key processes were: (1) teaching/learning; 

(2) enrollment management; (3) cultural environment; (4) strategic planning; 
(5) resource acquisition and development; (6) institutional management and 
governance; (7) research activity; (8) community outreach; (9) learning 
environment; and (10) communicating the institutional image. Of the key 
processes, the study found teaching/learning and enrollment management to 
have the highest agreement rate. Student retention data and exit examinations 
were identified as appropriate teaching and learning benchmarking measures; 
while retention data, student enrollment, and student satisfaction were cited 
as appropriate benchmarking measures for enrollment management. Twenty-one 
data tables summarize responses. The four appendixes include: the East 
Tennessee State University matrix; a copy of the letter of transmittal use; 
the questionnaire; and a directory of respondents. (Contains 40 references.) 
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ABSTRACT 



KEY PROCESS BENCHMARKING 
FOR CONTINUOUS IMPROVEMENT IN HIGHER EDUCATION 

by 

Robert Grisham Stewart 

The purpose of this study was to identify and verify the key 
processes and measures that can help higher education 
quality administrators implement a continuous improvement 
program and subsequent benchmarking process for their 
respective institutions. The benefits associated with 
benchmarking for continuous improvement are increased goal 
establishment and achievement and integration of proven 
practices . 

To identify these key processes the literature was reviewed 
to establish the elements of both benchmarking and 
continuous improvement. East Tennessee State University's 
Continuous Improvement Key Process Relationship Matrix was 
used as the basis for this research. 

To verify the ETSU Tern Key Processes and Measures survey 
research was conducted. A sample of forty-nine higher 
education institutions (unit of analysis) was drawn using a 
judgement (purposive) sampling technique. Each institution 
was represented by its resident quality administrator (unit of 
observation) who served as an informant for survey 
research. A forty-four item questionnaire was composed of 
ETSU'S Ten Key Processes and Thirty-Four Measures as 
extracted from the Matrix. A 53% total response (26 of 49) 
was obtained from administration of the questionnaire. 

Eight of the ten key processes and 16 of the 34 measures 
obtained an 80% or greater agreement response. The study 
findings indicate that these processes and measures can be 
used as a framework for implementing benchmarking for 
continuous improvement in higher education. 
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CHAPTER 1 



INTRODUCTION 



The Concept 



Higher Education 

Within the context of western society, higher education 
(higher learning) is defined as: 

instruction offered to persons of considerable 
intellectual maturity, usually requiring previous 
preparation through the secondary school; in terms of 
the institution common to the United States, higher 
education includes all education above the level of the 
secondary school given in colleges, universities., 
graduate schools, professional schools, technical 
institutes, teachers colleges, and normal schools. 

(Good, 1973, p. 282) 

One of the oldest continuously operating universities, 
Cairo's University of Al-Azhar, was founded in 970 (Millet, 
1969) . Although the Arabs have been credited as the first 
to establish institutions of higher education, historians 
believe the western forerunners of modern universities were 
the University of Paris and the University of Bologna which 
were established in Europe during the 1100 's (Millett, 

1969) . Harvard University, the first and oldest operating 
institution of higher education in the United States, was 
originally chartered by the state of Massachusetts as 
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Newtowne College in 1636 (Millett, 1969) . Over the last 360 
years the number of higher education institutions in the 
United States has grown to approximately 3665 schools 
(Rodenhouse, 1995). 

According to Bemowski (1991) , the two main functions of 
higher education institutions are "to educate and to 
generate knowledge" (p. 37) . However, many of those 
associated with higher education acknowledge that these 
functions are not being adequately accomplished and advocate 
that higher education must improve if it is to remain viable 
(Kaufman & Zahn, 1993; Lewis & Smith, 1994; Seymour, 1992) . 
In effectively summarizing the need for improvement in 
higher education, Lewis and Smith (1994) state: 

the environment of higher education is changing and 
competition for both students and funds will continue 
to increase, at a time when we are going to have to 
accomplish more with less. The result is that colleges 
and universities in the coming century will not be the 
same as they are today. Thus, the question that must 
be addressed is how we as members of the academy will 
respond to these (and related) trends. Will we respond 
in a proactive manner and initiate positive, quality- 
focused, learner-centered programs, or will we respond 
in a defensive manner, attempting to preserve the past 
at the expense of the future? (p. x) 
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Continuous Improvement 

Continuous improvement (Cl) may serve as one possible 
answer to the question posed by Lewis and Smith. According 
to Leibfried and McNair (1992), the philosophy of continuous 
improvement is a "never-ending-quest to be just a little bit 
better, every day, in every activity" (p. 97) . The 
following names have been used to label this philosophy: 
total quality management (TQM) , continuous quality 
improvement (CQI) , quality improvement process (QIP) , 
quality management (QM) , and Kaizen. Quality is the only 
term explicitly expressed in all of these, except Cl and 
Kaizen, where its meaning is implied. 

Within the context of higher education, Hittman (1993) 
operationally defines quality as "a continuous effort by all 
members of an organization to meet students' and other 
interested parties' needs and expectations" (p. 78) . 

However, as Bradley (1993) concludes, "What was quality in 
the past is not quality today, and what is quality today 
will not suffice as quality in the future" (p. 3) . Its only 
constant is the meeting of needs and expectations. 

The idea of controlling quality began with the first 
artisan who was responsible for performing all tasks 
relative to a product (Bradley, 1993) . With the emergence 
of the Industrial Revolution, quality control tasks became 
the responsibility of full time inspectors and supervisors 
(Bradley, 1993) . Frederick W. Taylor (1856-1915) , Henry L. 




Gantt (1861-1919), Frank B. Gilbreath (1886-1924), Lillian 
M. Gilbreath (1878-1972) and Harrington E. Emerson (1853- 
1931) developed a system for increasing productivity called 
scientific management, or Taylorism. Scientific management 
principles greatly emphasized production at the expense of 
quality, which gave American industries the ability to 
produce large quantities of materials. However, by 1915 the 
popularity of the scientific management movement began to 
decline because of its deemphasis of quality and its 
disregard for the well being of laborers (Bradley, 1993) . 

Walter Shewhart , W. Edwards Deming, Armaund Fiegenbaum, 
Joseph Juran, and Philip Crosby transformed the principles 
of scientific management into the philosophy of quality 
management. In 1924, Shewhart introduced a charting system 
which utilized control limits combined with statistical 
probability to predict the production of inferior products. 
In the 1930' s, Shewhart collaborated with Deming to design a 
quality management system which views the production process 
holistically. At the heart of this management system lies 
Deming' s fourteen points, the first of which is to create a 
constancy of purpose or according to Lewis and Smith (1994) , 
continuous improvement. 

Just as education adopted the principles of scientific 
management from industry in the 20th century (Hittman, 

1993), many have advocated that higher education must adopt 
the principles of quality management (Bradley, 1993; Kaufman 
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& Zahn, 1993; Lewis & Smith, 1994; and Seymour, 1992). The 
adaptation of these principles from the private sector has 
already begun. Although Deming' s fourteen points were 
originally written for business, they have been translated 
for education by Bradley (1993) , Kaufman and Zahn (1993) , 
and Wilcox (1992) . Hull (1991) effectively summarizes the 
need for adopting the philosophy of continuous improvement 
for higher education: 

Academic life in America today exists in a world with 
too many schools and too few students, too many fixed 
costs and too few discretionary dollars, too many 
competitors and too few supporters. In such a world, 
survival does belong to the fittest, which will be 
those institutions imbued with a passion for quality 
that extends to every member of the community, faculty 
included, (p. 227) 

Benchmarking 

According to Leibfried and McNair (1992), "Benchmarking 
is based in the philosophy of continuous improvement" 

(p. 18) . Therefore, it can serve as a catalyst for 
organizational acceptance of a continuous improvement 
program because it evaluates existing performance, 
establishes future goals, and targets improvements. 
(Leibfried & McNair, 1992) . Within the context of higher 
education, Dabney, Lassila, and Collins (1995) offer the 
following constitutive definition of benchmarking: 
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(1) a quantitative process for measuring, comparing 
and, assessing productivity, performance, and goals 
relative to other institutions; (2) a qualitative 
process for identifying best practices to improve one's 
own practices; (3) a method for opening communication 
with other universities and sharing information, (p. 2) 
Benchmarking first occurred during the scientific 
management movement of the 1800' s when Frederick Taylor 
advocated using a series of checks to compare performance 
(Watson, 1993). Since that time, benchmarking's complexity 
and applications have increased. In the early 1980 's 
benchmarking, then known as reverse engineering, evolved 
into competitive benchmarking. In early 1989, process 
benchmarking became popular and served as the basis for 
strategic benchmarking. The next step in the evolutionary 
process of benchmarking will be its implementation on a 
global scale (Watson, 1993) . Clark (1993) effectively 
summarizes the need for benchmarking in higher education: 
Universities can no longer experience the luxury of 
claiming their ranks contingent upon historical 
contributions. Institutions of higher education must 
indulge in strategies such as benchmarking in order to 
further develop instructional paradigms that serve to 
enrich their contribution, (p. 9) 



O 

ERIC 



17 



7 



The Problem 



Statement 

Within the concept of benchmarking for continuous 
improvement in higher education the author identified the 
following problem: 

Higher education quality administrators are uncertain 
what key processes and associated measures are suitable for 
implementing a continuous improvement program and subsequent 
benchmarking process within their respective institutions. 

Definition 

Within the context of the problem the following units 
and variables were operationally defined: 

Unit of analysis. Institutions of higher education were 
the units of analysis of the study. An institution of 
higher education is any public or private university or 
college that conducts purposeful teaching and learning 
utilizing an organized curriculum of instruction beyond the 
level of the secondary school (Butts, 1969) . 

Unit of observation. Higher education quality 
administrators were the units of observation of the study. 

A higher education quality administrator is any person 
charged with improving a process or service of his or her 
respective institution (Hales, 1996) . 
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Variable of uncertainty. The quality or state of not 
being clearly identified or defined is the operational 
definition of uncertainty (McKechnie, 1983). 

Variable of suitability. The quality or state of being 
appropriate, proper, or fitting is the operational 
definition of suitability (McKechnie, 1983). 

Significance 

The benefits resulting from benchmarking for continuous 
improvement in higher education are increased goal 
establishment and achievement and an increased 
competitiveness through the integration of proven practices 
(Camp, 1992) . However, the uncertainty regarding the 
selection of key processes restricts attainment of these 
benefits. If key processes were identified, then the 
ability of quality administrators to make these benefits a 
reality for their respective institutions would be enhanced. 

The Study 

Purpose 

The objectives of the study were to (1) identify; and 
(2) verify the key processes and associated measures that 
can help higher education quality administrators implement a 
continuous improvement program and subsequent benchmarking 
process for their respective institutions. 
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Questions 

To accomplish the objective of identifying these key 
processes and associated measures, the following questions 
were defined: 

1. What are the elements of continuous improvement as 
stipulated by the literature? 

2. What are the elements of benchmarking as stipulated by 
the literature? 

The following information was obtained from 
investigating the preceding questions: 

1. The elements of continuous improvement are a leadership 
council, a mission statement, a vision statement, a 
values statement, and a goals statement. 

2. The elements of benchmarking are a process-based 
management approach, key business processes, key process 
measures, a benchmarking team, process owners, a list of 
benchmarking partners or information sources, the 
documentation of internal products and processes, and an 
action plan. 

The objective of this research was to verify East 
Tennessee State University's Continuous Improvement Key 
Process Relationship Matrix as a means for initiating 
continuous improvement and benchmarking in higher education. 
1. How do higher education quality administrators feel 
about the suitability of East Tennessee State 
University' s Ten Key Processes and Thirty-Four Measures 
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as obtained from the Continuous Improvement Key Process 
Relationship Matrix? 

a. Which key processes and measures do quality 
administrators agree are suitable? 

b. Which key processes and measures do quality 
administrators agree are unsuitable? 

2. What "other" key processes and measures do quality 
administrators propose for inclusion? 

These questions were investigated using a survey 
research method outlined in Chapter 3 of the study. The 
information resulting from this investigation is discussed 
in Chapter 4 of the study. 

Vocabulary 

The following terms have been defined according to 
their usage within the context of this study: 

Higher education benchmarking. According to Dabney, 
Lassila, and Collins (1995) , it is " (1) A quantitative 
process for measuring, comparing, and assessing 
productivity, performance, and goals relative to other 
institutions; (2) a qualitative process for identifying best 
practices to improve one's own practices; (3) a method for 
opening communication with other universities and sharing 
information" (p. 2) . 
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Higher education institution (unit of analysis) . Any 
public or private university or college that conducts 
purposeful teaching and learning utilizing an organized 
curriculum of instruction beyond the level of secondary 
school (Butts, 1969) . 

Higher education quality administrator (unit of 
observation) . Any person charged with improving a process or 
service of his or her respective institution (Hales, 1996). 

Key measures. The essential measurements for assessing 
the performance of a process which may include in-process 
and post-process measurements (Camp, 1995) and include both 
indicators of performance and progress. According to 
Fischer (1994), a "performance indicator, or benchmark, is 
one criterion underlying successful program or service 
performance" (p . S-5). Progress indicators measure the 
amount of perceived accomplishment within the context of 
creative activities such as painting, dancing, and musical 
composition (Hales, 1996). 

Key processes. Kessler (1995) defines them as "The most 
critical processes to customer satisfaction and the survival 
of the organization" (p. 146). 
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Key (critical) success factors. Watson (1993) defines 
them as "those few activities where satisfactory performance 
is essential in order for a business to succeed" (p. 260) . 

Quality. Hittman (1993) defines it as "a continuous 
effort by all members of an organization to meet students' 
and other interested parties' needs and expectations" 

(p. 78) . 

Sub-processes (work processes) . Camp (1995) defines 
them as processes "that are entirely within the control of a 
single function and can be changed as such" (p. 7) . 

Suitability. The quality or state of being appropriate, 
proper, or fitting (McKechnie, 1983) with regard to key 
processes and associated measures for implementing 
benchmarking for continuous improvement in higher education. 

Uncertainty . The quality or state of not being clearly 
identified or defined (McKechnie, 1983) with regard to key 
processes and associated measures for implementing 
benchmarking for continuous improvement in higher education. 

Zero-based institution. An institution that has no 
existing quality system (Fisher, 1995) . 
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Assumptions 

Survey research. A survey research plan was used to 
collect primary data applicable to a problem solution. It 
was assumed that survey research was representative of a 
problem solution. 

Target population. The target population of the study 
was those colleges and universities (both zero-based 
institutions and committed to quality institutions) that are 
uncertain what key processes and associated measures are 
suitable for implementing benchmarking for continuous 
improvement . It was assumed that the target population was 
representative of definition validity and subsequently the 
research problem. 

Access population. The access population of the study 
was the 220 universities and colleges of the 1995 American 
Society for Quality Control's Directory of Higher Education 
(Calek, 1995) . It was assumed that the access population 
was representative of the target population. 

Sample units. A judgement (purposive) nonprobability 
sampling technique was used to draw sample units from the 
access population of the study. It was assumed that the 
sample units were representative of the access and 
subsequent target populations. 
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Sample size. An experienced researcher's judgement and 
two required sample characteristics were used to select 49 
institutions of higher education (22%) from the 220 access 
population elements of the study. It was assumed that the 
sample size was representative of population validity. 

Questionnaire . The author developed a forty- four item 
questionnaire in order to measure the suitability of East 
Tennessee State University's Ten Key Processes and Thirty 
Four Measures with regard to higher education quality 
administrators. It was assumed that the questionnaire was 
representative of measurement validity. 

Total response. A 53 percent total response was 
obtained from administration of the questionnaire... It was 
assumed that the total response was representative of the 
sample units. 

Item response. It was assumed that an item response 
was representative of a truthful response. 

Item agreement response. An 80 percent item agreement 
response was used to represent the suitability of a key 
process or associated measure. The author acknowledges that 
a citation advocating an agreement percentage for a decision 
making group representative of suitability was not 
available. Therefore, it was assumed that an item agreement 
response rate of 80 percent represents suitability. 
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Limitations 

Inclusions . Quality administrators were asked to 
propose key processes and measures for inclusion in the ETSU 
Continuous Improvement Key Process Relationship Matrix. 
However, the study was limited to the collection of 
proposals only. The determination of which proposed items 
are suitable for inclusion was not an objective of this 
study and should be obtained through further research. 
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CHAPTER 2 



LITERATURE REVIEW 
The Elements 

Continuous Improvement 

The following elements have been identified by the 
author as being fundamental to a continuous improvement 
framework : 

Leadership council. The creation of a leadership 
council is an important step in the development of 
continuous improvement program (Lewis & Smith, 1994). 
Consisting of senior leaders representing the major areas of 
an institution, the council is responsible for all aspects 
of a continuous improvement program (Lewis & Smith, 1994) . 
The element of leadership is central to the concept of 
continuous improvement (Deming, 1986; Harris, 1992; Lewis & 
Smith, 1994; Seymour, 1992). Therefore, the first element 
of continuous improvement is a leadership council. 

Mission statement. An institution's mission is its 
fundamental reason for existence (Fisher, 1995) . The 
leadership council's first critical task is the 
identification and definition of an institutional mission 
(Lewis & Smith, 1994) . According to Fisher (1995), a 
mission statement officially acknowledges an institution's 
purpose via a published document which is "shared with 

16 
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faculty, staff, students, suppliers, customers, and the 
community" (p. 237). An institution's dedication to and 
comprehension of a mission statement provides a decision 
making framework within a continuous improvement program 
(Lozier & Teeter, 1993) . Additionally, the development of 
an institution's mission statement serves to increase 
cooperation among members (Lewis & Smith, 1994) . Therefore, 
the second element of continuous improvement is a mission 
statement . 

Vision statement . An institution's vision, as defined 
by its leadership council, describes what it will resemble 
upon achievement of its goals (Lozier & Teeter, 1993) . A 
vision statement officially acknowledges an institution's 
vision through a published document which is shared 
throughout the institution and community (Fisher, 1995) . 

The absence of a vision statement will limit the 
advancements of an institution's continuous improvement 
program (Lozier & Teeter, 1993) . Therefore, the third 
element of continuous improvement is a vision statement. 

Values statement . An institution's values are the 
beliefs which provide a decision making framework for 
accomplishment of its mission (Lewis. & Smith, 1994) . 
According to Fisher (1995) , a values statement officially 
acknowledges an institution's beliefs through a published 
document which is "shared with faculty, staff, students, 
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customers, suppliers, and the community" (p. 240) . An 
institution's values are fundamental to the success of its 
mission and its subsequent continuous improvement program 
(Lewis & Smith, 1994) . Therefore, the fourth element of 
continuous improvement is a values statement . 

Goals statement. An institution's goals (critical 
success factors) represent the strategies for obtaining both 
short-term and long-term results (Fisher, 1995) . A goals 
statement officially acknowledges an institution's goals 
through a published document which is shared throughout the 
institution and community (Fisher, 1995) . Goals provide a 
framework that connect a continuous improvement program's 
strategic mission, vision, and values with departmental 
tactical activities (Lewis & Smith, 1994) . Therefore, the 
fifth element of continuous improvement is a goals 
statement . 

Benchmarking 

According to Spendolini (1992) , benchmarking is a 
"structured process" (p. 38) . Therefore, its framework can 
be represented by a process model (Balm, 1992; Bogan & 
English, 1994; Boxwell, 1994; Camp, 1989, 1995; Karlof & 
Obstrum, 1995; Leibfried & McNair, 1992; and Watson, 1992, 
1993) . However, the existence of multiple process models 
creates uncertainty in determining which one truly 
represents the benchmarking process. To resolve this issue 
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Spendolini (1992) developed a generic five stage 
benchmarking process model that is a synthesis of twenty- 
four existing models. The five stages of Spendolini 's Model 
(1992) determine what to benchmark, form a benchmarking 
team, identify benchmarking partners, collect and analyze 
benchmarking information, and take action. According to 
Camp (1995), two distinct benchmarking processes exist: the 
users process (i.e. Spendolini 's five stage model) and the 
management process. A benchmarking team follows the users 
process in order to successfully complete a benchmarking 
project. Conversely, the management process supports the 
benchmarking team by ensuring an atmosphere conducive to 
business process improvement (Camp, 1995) . 

Because benchmarking is an advanced management tool, an 
organization must determine if its management framework has 
sufficient maturity to support benchmarking processes 
(Watson, 1993) . If the outcomes of a benchmarking process 
are to be useful, they must be linked to an organization's 
mission, vision, values, and goals (Camp, 1995) . Therefore, 
a continuous improvement program must be in place to support 
benchmarking . 

Camp (1995) acknowledges that two approaches for a 
managing the benchmarking process are problem-based and 
process-based management. Initially, the management of 
benchmarking activities were ad hoc in nature, and responded 
to problems as they occurred. According to Camp (1995), 
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this problem-based management process was maintained by 
"customer feedback, the cost base, a desire to reduce error 
rates, high asset levels, the need to improve cycle times, 
or the like" (p. 5) . 

More recently organizations have recognized the need to 
manage benchmarking in order to generate improvements. 
Process-based benchmarking management focuses on improving 
those business processes that are related to goals which are 
vital to the success of an organization (Camp, 1995) . 
According to Watson (1993) , "Business processes are logical 
combinations of people, equipment, materials, and methods 
organized into work activities that produce desired outputs" 
(p. 56) . Camp (1995) concludes that: 

It is the focus of benchmarking on the business process 
that has come to be accepted as the correct approach 
for benchmarking. Thus it is process benchmarking, not 
problem-based benchmarking, that is needed, (p. 7) 
Therefore, the first element of benchmarking is a process- 
based management approach. 

The first stage of Spendolini's (1992) Benchmarking 
Process Model (users process) determines what to benchmark. 

A variety of things can be selected for benchmarking 
however, because organizations have limited resources they 
should focus on the processes that offer the greatest return 
on investment (Watson, 1993) . 




31 



21 



Watson (1993) advocates that key business processes: 

are prime process candidates for benchmarking because 
they have a broader scope than specific business 
practices and are essential to achieving the critical 
success factors of an organization, (p. 56) 

Therefore, the second element of benchmarking are key 
business processes. 

Key process performance measures are the essential 
measurements for assessing the performance of a key process. 
According to Camp (1995) there are two types of process 
measures: post-process and in-process. Utilizing both post- 
process and in-process measures will increase the chances of 
recognizing the essential improvements that will close 
process performance gaps (Camp, 1995) . Therefore, the third 
element of benchmarking are key process measures . 

The second stage of Spendolini's (1992) Benchmarking 
Process Model forms a benchmarking team. Due to the amount 
of work involved with benchmarking, a team is essential for 
ensuring the success of a project (Spendolini, 1992) . 
Therefore, the fourth element of benchmarking is a 
benchmarking team. A major task in forming a benchmarking 
team is to determine its members. Curtice (1995), DeToro 
(1995), and Watson (1993; 1992) advocate that process owners 
should be assigned to a benchmarking team. According to 
Watson (1993) , a process owner is "the individual who 
possess managerial control over a particular business 
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process" (p. 261) . Therefore, the fifth element of 
benchmaring are process owners. 

The third stage of Spendolini's (1992) Benchmarking 
Process Model identifies benchmarking partners. The 
selection of the appropriate partners is essential to the 
success of a benchmarking project (Spendolini, 1992) . 
Therefore, the sixth element of benchmarking is a listing of 
benchmarking partners or information sources. 

The fourth stage of Spendolini's (1992) Benchmarking 
Process Model is the collection and analysis of benchmarking 
information. An organization must document its own 
organizational performance prior to external benchmarking 
(Spendolini, 1992). Therefore, the seventh element of 
benchmarking is documentation of internal products and 
processes . 

The fifth stage of Spendolini's (1992) Benchmarking 
Process Model is to take action. The major task of a 
benchmarking team is the identification of improvement 
opportunities (Spendolini, 1992) . Once these improvements 
have been identified, a benchmarking team must decide on 
what actions will be taken to implement improvements 
(Spendolini, 1992) . Therefore, the eighth element of 
benchmarking is an action plan. 

In summary, continuous improvement, a process-based 
management approach, key business processes, key process 
measures, a benchmarking team, process owners, a listing of 
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benchmarking partners or information sources, the 
documentation of internal products and processes, and an 
action plan are all necessary to successfully implementing 
benchmarking for continuous improvement in higher education. 

Identification of key processes and associated measures 
are the essential elements by which institutions of higher 
education can implement benchmarking (International Forum 
for Quality in Higher Education, 1995) . This research 
focused on these elements to provide a basis for initiating 
the benchmarking journey in higher education. 




CHAPTER 3 



METHODOLOGY 
The Population 

Target . The target population of the study was those 
colleges and universities (both zero-based institutions and 
committed to quality institutions) that are uncertain what 
key processes and associated measures are suitable for 
implementing benchmarking for continuous improvement. 

Access . The access population of the study was the 220 
universities and colleges of the 1995 American Society for 
Quality Control Directory of Higher Education (Calek, 1995) . 

The Sample 

Those quality administrators possessing expert 
knowledge of key processes and measures of higher education 
were the desired sample units of the survey. Because of the 
specific nature of this attribute a nonprobability judgement 
(purposive) technique was utilized to draw the sample units 
from the access population. This technique stipulates that 
an experienced researcher perform the drawing by judging 
population elements against attributes of the desired sample 
units. Dr. James A. Hales was selected to perform the 
drawing due to his inherent familiarity with institutions 
implementing quality improvement programs. The number of 
years an institution has been implementing quality 
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improvement programs and the key informant positions of 
president, vice president, provost, and director of quality 
were selected as judgement attributes of the desired sample 
units. Because a hypothetical construct relating attribute 
to knowledge was not available it was necessary to assume 
that the sample judgment attributes would provide units 
knowledgeable of key processes and measures of higher 
education. Forty-nine institutions of higher education were 
selected to represent the sample units of the survey. 

The Questionnaire 



Construction 

The suitability of the ETSU Continuous Improvement Key 
Process Relationship Matrix was assessed with a forty-four 
item, self -administered questionnaire (Appendix C) developed 
for use in this study. The response items were taken 
directly from the East Tennessee State University Key 
Process Relationship Matrix (Appendix A) and consisted of 10 
key processes and 34 associated measures. The rating scale 
was constructed by assigning a simple dichotomy of agree or 
disagree to each item. An open ended format was used to 
provide space for proposing "other" items if participants 
desired . 
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Communication 

A letter of transmittal (Appendix B) was composed for 
the survey portfolio of the study. According to Erdos 
(1970), a letter of transmittal should (1) be grammatically 
correct; (2) not exceed one page in length; (3) be written 
in the second person; (4) contain a salutation with the 
recipient's name and title; (5) request assistance from the 
recipient; (6) state the purpose of and describe the 
incentive for responding if any; (7) explain the purpose and 
significance of the research; (8) explain how the recipient 
may benefit from the research; (9) state the amount of time 
required to complete the survey; (10) state the level of 
difficulty in responding to the survey; (11) state the 
degree of respondent confidentiality; (12) state the 
deadline for responding if any; (13) state the method (s) for 
returning the survey; (14) thank the recipients for their 
efforts; (15) acknowledge the importance of the sender; (16) 
acknowledge the importance of the sender's organization; 

(17) contain a closure with the sender's signature; (18) be 
separate from the questionnaire; and (19) avoid inducing 
biased responses from the recipients. These stipulations 
were met by the transmittal letter (Appendix B) that was 
composed for the survey portfolio of the study. 
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Distribution 

On January 19, 1995 the survey portfolios consisting of 

a transmittal letter, questionnaire, return envelope, return 
postcard, and return postage (for foreign participants only) 
were mailed. 

The Data 



Tabulations 

Upon receipt of a completed questionnaire the author 
reviewed it to ensure that the respondents had interpreted 
the questionnaire correctly and responded accordingly. 

Next, the questionnaire was coded by assigning an 
identification number. Finally, a simple tally, counting 
the number of responses to an item and placing them in a 
frequency distribution, was performed. 

Calculations 

Total response. A 53 percent total response (26 out of 
49) was obtained from the administration of the 
questionnaire. The percentage of total response was 
calculated by dividing the number of questionnaires returned 
by the number of sample units. 

Item agreement response. The percentage of respondents 
agreeing with an item was calculated by dividing the number 
of item agreement responses by the number of total 




responses . 
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Item disagreement response. The percentage of 
respondents disagreeing with an item was calculated by 
dividing the number of item disagreement responses by the 
number of total responses. 

Item nonresponse. The percentage of respondents not 
responding to an item was calculated by dividing the number 
of item nonresponses by the number of total responses. 

Comparisons 

An item agreement response percentage of 80% or greater 
was representative of suitability. All item response 
averages were compared with this 80% suitability rate. 




CHAPTER 4 



DISCUSSION 



The Information 



Quantitative 

The following information was obtained from the 
survey research. The findings reveal that suitability 
was achieved for eight of ETSU's Ten Key Processes. This 
indicates that these eight processes can serve as a 
practical framework for implementing continuous 
improvement and a subsequent benchmarking process in 
higher education. A discussion of each key process and 
associated findings from this study now follows: 
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The key process of Teaching and Learning represents 
the collective activities of curriculum development, 
academic support for instruction, student life program 
development, professional development and evaluation, and 
continuing studies. Table 1 shows that 80% agree this is a 
suitable process for implementing benchmarking for 
continuous improvement in higher education. Appropriate 
measures that may be used as performance indicators are: 
student exit examinations with 88% agreeing, student 
retention data with 84% agreeing, and student entrance 
examinations with 76% agreeing. 



TABLE 1 

ITEM RESPONSES: TEACHING AND LEARNING 



Item 




Agree 


Response 

Disagree 


None 


Process 


Teaching/Learning 


0 . 88 


0 . 00 


0 . 12 


Measure 


Student 


exit examinations 


0.88 


0 . 08 


0 . 04 


Student 


retention data 


0 . 84 


0 . 08 


0 . 08 


Student 


entrance examinations 


0 . 76 


0 . 20 


0 . 04 


Note. An item 


agreement percentage 


of 80% 


or greater 


is 



representative of suitability. 
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The key process of Enrollment Management represents the 
collective activities of recruitment, admission, financial 
aid, orientation, advisement, registration, scheduling, and 
housing. Table 2 shows that 88% agree this is a suitable 
process for implementing benchmarking for continuous 
improvement in higher education. Appropriate measures that 
may be used as performance indicators are: retention data 
with 92% agreeing, student enrollment and student 
satisfaction both with 88% agreeing, and admissions growth, 
diversity data, and student success, each with 80% agreeing. 



TABLE 2 

ITEM RESPONSES : ENROLLMENT MANAGEMENT 



I tern 


Agree 


Response 

Disagree 


None 


Process 


Enrollment management 


0 .88 


0 . 00 


0.12 


Measure 


Retention data 


0 . 92 


0.00 


0.08 


Student enrollment 


0 . 88 


0 .00 


0 . 12 


Student satisfaction 


0 .88 


0 . 04 


0 . 08 


Admissions growth 


0 .80 


0 . 08 


0 . 12 


Diversity data 


0 .80 


0 . 08 


0 . 12 


Student success 


0 .80 


0 . 04 


0 . 16 


Note. An item agreement percentage 


of 80% 


or greater 


is 



representative of suitability. 
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The key process of Enhancement of the Cultural 
Environment represents the collective activities of cultural 
enrichment, promotion of regional culture, teaching cultural 
diversity, and international studies. Table 3 shows that 
80% agree this is a suitable process for implementing 
benchmarking for continuous improvement in higher education. 
However, direct measures were more elusive, perhaps because 
enhancement of the cultural environment is difficult to 
measure on an absolute scale. The study findings are that 
72% agree with the number of persons attending, 68% agree 
with number of events, and 56% agree with peer institution 
comparisons. Further study is recommended to identify 
performance indicators rather than absolute measures. 

TABLE 3 

ITEM RESPONSES: ENHANCEMENT OF THE CULTURAL ENVIRONMENT 



Item 




Agree 


Response 

Disagree 


None 


Process 


Cultural environment 


enhancement 


0 . 80 


0 . 12 


0.08 


Measure 


Number of persons 


attending 


0 . 72 


0 . 16 


0 . 12 


Number of events 




0 .68 


0 .16 


0 . 16 


Peer institution 


comparisons 


0 . 56 


0.28 


0 . 16 



Note . An item agreement percentage of 80% or greater is 
representative of suitability. 
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The key process of Strategic Planning, Development, and 
Budgeting collectively represents long-term, strategic, 
master, and outcome assessment activities. Table 4 shows 
that 84% agree this is a suitable process for implementing 
benchmarking for continuous improvement in higher education. 
Goal achievement, with 88% agreeing, performance on outcome, 
with 84% agreeing, and student and alumni surveys, with 84% 
agreeing, received the highest item agreement response 
rates. Resource availability, with 72% agreeing, and peer 
comparisons, with 68% agreeing, are also indicators. 



TABLE 4 

ITEM RESPONSES: STRATEGIC PLANNING/DEVELOPMENT/BUDGETING 



Item 


Agree 


Response 

Disagree 


None 


Process 








Strategic planning development 


0 . 84 


0.08 


0 . 08 


and budgeting 








Measure 








Goal achievement 


0 . 88 


0 .00 


0 . 12 


Student/alumni surveys 


0 . 84 


0 . 04 


0 . 12 


Performance/outcome measures 


0 . 84 


0 .00 


0.16 


Resource availability 


0 . 72 


0 . 16 


0.12 


Peer institution comparisons 


0 . 68 


0 . 16 


0 . 16 


Note . An item agreement percentage 


of 80% 


or greater 


is 



representative of suitability. 
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The key process of Resource Acquisition, Development, 
Management, and Accountability collectively represents 
fiscal, human resource, and physical activities. Table 5 
shows that 84% agree this is a suitable process for 
implementing benchmarking for continuous improvement in 
higher education. Appropriate measures achieving agreement 
are: peer institution comparisons, with 88% agreeing, 
faculty and staff satisfaction, with 84% agreeing, and 
requested vs. achieved, with 80% agreeing. 

TABLE 5 

ITEM RESPONSES: RESOURCE ACQUISITION/DEVELOPMENT 
MANAGEMENT AND ACCOUNTABILITY 



Item 


Agree 


Response 

Disagree 


None 


Process 


Resource acquisition, development 0.84 
management and accountability 


0 . 04 


0 . 12 


Measure 


Peer institution comparisons 


0 . 88 


0 . 04 


0 . 08 


Faculty/staff satisfaction 


0 . 84 


0 . 04 


0.12 


Requested vs . achieved 


0 . 80 


0 . 04 


0 . 16 


Note. An item agreement percentage 


of 80% 


or greater 


is 



representative of suitability. 
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The key process of Institutional Management and 
Governance represents the collective activities of 
communications, data management, internal governance, and 
external governance. Table 6 shows that 84% agree this is 
a suitable process for implementing benchmarking for 
continuous improvement in higher education. Appropriate 
measures and performance indicators are faculty and staff 
satisfaction, with 92% agreeing, program accreditation, with 
88% agreeing, mission and goal accomplishment, with 84% 
agreeing, and peer institution comparisons, with 76% 
agreeing . 



TABLE 6 

ITEM RESPONSES: INSTITUTIONAL MANAGEMENT /GOVERNANCE 



Item 


Agree 


Response 

Disagree 


None 


Process 








Institutional management 


0 . 84 


0 . 08 


0 . 08 


governance 








Measure 








Faculty staff /satisfaction 


0 . 92 


0 . 00 


0.08 


Program accreditation 


0.88 


0.00 


0 . 12 


Mission/goal accomplishment 


0 . 84 


0 . 04 


0 . 12 


Peer institution comparisons 


0 . 76 


0 . 12 


0.12 




Note . An item agreement percentage of 80% or greater is 
representative of suitability. 
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The key process of Research, Scholarship, and Creative 
Activity includes research grant or contract development. 
Table 7 shows that 80% agree this is a suitable process for 
implementing benchmarking for continuous improvement in 
higher education. These processes present difficulties in 
terms of absolute measurement as indicated by the item 
agreement response of 56% for juried shows and performances 
and 68% for the number of publications and presentations. 
Further study is recommended to identify suitable indicators 
for assessing creative processes in higher education. 



TABLE 7 

ITEM RESPONSES: RESEARCH/SCHOLARSHIP/CREATIVE ACTIVITY 







Response 




Item 


Agree 


Disagree 


None 


Process 








Research/ scholarship/creativity 


0 . 80 


0 . 04 


0.16 


Measure 








Peer institution comparisons 


0 . 84 


0 . 04 


0 . 12 


Internal support 


0 . 76 


0 . 08 


0 . 16 


Amount of dollars generated 


0 . 76 


0 . 08 


0 . 16 


Number of grants received 


0 . 72 


0 . 12 


0 . 16 


Number of publications 


0 . 68 


0.16 


0 . 16 


Juried shows/performances 


0 . 56 


0 . 16 


0.28 


Note. An item agreement percentage 


of 80% 


or greater 


is 



representative of suitability. 
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The key process of Service (Community Outreach) 
represents the collective activities of community service, 
foreign institution partnerships, and alumni relations. 
•Table 8 shows that 84% agree this is a suitable process for 
implementing benchmarking for continuous improvement in 
higher education. However, absolute measures of this 
process are difficult to identify with only 60% agreeing 
with peer comparisons and 76% with user satisfaction. An 
area for further investigation is also suggested here to 
identify performance indicators that may be more helpful in 
implementing the key process of service within the context 
of continuous improvement in higher education. 



TABLE 8 

ITEM RESPONSES: SERVICE (COMMUNITY OUTREACH) 



Item 


Agree 


Response 

Disagree 


None 


Process 








Service (Community Outreach) 


0.84 


0 . 04 


0 . 12 


Measure 








User satisfaction 


0 . 76 


0.08 


0 . 16 


Peer institution comparisons 


0 . 60 


0.20 


0.20 


Note. An item agreement percentage 


of 80% 


or greater 


is 



representative of suitability. 
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The key process of Developing and Maintaining a 
Learning Environment represents the collective activities of 
student services, career development and placement, special 
programs, and intercollegiate athletics. Table 9 shows that 
76% agree that this is a suitable process for implementing 
benchmarking for continuous improvement in higher education. 
This process did not achieve the established suitability 
rate of 80%. However, student and alumni satisfaction 
achieved an 84% agreement response as a measure of this 
process. Further study is recommended to determine what 
modifications are required to improve the suitability of 
this process. 



TABLE 9 

ITEM RESPONSES: DEVELOPING/MAINTAINING LEARNING ENVIRONMENT 



Item 


Agree 


Response 

Disagree 


None 


Process 

Developing/maintaining a 


0 . 76 


0 . 08 


0 . 16 


learning environment 
Measure 

Student /alumni satisfaction 


0 . 84 


0 . 04 


0 . 12 



Note . An item agreement percentage of 80% or greater is 
representative of suitability. 
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The key process of Communicating the Institutional 
Image represents the collective activities of media 
relations, campus publications, audiovisual productions, 
community relations, marketing and promotions, internal 
communications, and institutional radio. Table 10 shows 
that 76% agree this is a suitable process for implementing 
benchmarking for continuous improvement in higher education. 
Neither the process nor the measure, (peer institution 
comparisons with 64% agreeing) , achieved the established 
suitability rate of 80%. Further study is recommended to 
determine what modifications are required to improve the 
suitability of this process. 

TABLE 10 

ITEM RESPONSES: COMMUNICATING THE INSTITUTIONAL IMAGE 



Item 


Agree 


Response 

Disagree 


None 


Process 

Communicating the institutional 


0 . 76 


0 .08 


0 . 16 


image 

Measure 

Peer institution comparisons 


0 . 64 


0 . 16 


0.20 



Note . An item agreement percentage of 80% or greater is 
representative of suitability. 
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Qualitative 

The following information was provided by quality 
administrators in response to the second survey research 
question: "What other key processes and measures do you 

propose for inclusion?" 

TABLE 11 

ITEM RESPONSES: OTHER TEACHING AND LEARNING MEASURES 

Amount of student participation in classroom learning system 
(Does the instructor ask for student input?) 

Amount of curriculum compliance with accreditation standards 
Amount of time and budget for staff and faculty development 
Amount of academic support for instruction (survey) (1.4) 
Amount of classroom improvements (1.6) 

Amount of faculty development (in number of hours) 

Amount of faculty satisfaction 

Amount of team utilization in the classroom 

Amount of curriculum development articulation 

Approval of curriculum developments by advisory board 

Amount of development /revision/benchmarking (1.1) [3] 

Benchmarks of best-in-class in each area (1.4-1. 7) 

Cycle time of new program development process 
Final examination data from disciplines 
High school data: class ranking, grade point average 
course work 

(table continues) 
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TABLE 11 (continued) 

ITEM RESPONSES: OTHER TEACHING AND LEARNING MEASURES 

Number of faculty teaching awards 

Number of professionally accredited programs 

Number of students per class (average) 

Pre/post assessment of writing portfolios, critical thinking 
problem solving [2] 

Percent of students scoring above 50th & 80th percentiles on 
Engineer in Training (EIT) Examination 
Graduate Record Examination (GRE) 

Graduate Management Admission Test (GMAT) 

Medical College Admission Test (MCAT) 

Percentage of students attending graduate school 
Quality of capstone courses (1.3) 

Quality of faculty performance (by student ratings) 

Quality of alumni performance in jobs/graduate schools 
Quality of student self-assessments/reports (1.3) 

Quality of faculty evaluations (by trend analysis) (1.7) 
Quality of faculty credentials 
Ratio of students-to-faculty 

Rate of graduation by discipline, ethnicity, and time [2] 
Rate of graduate employment [3] 

Rate of faculty turnover 

Standardization of examinations (if possible) 

Student retention data (qualified by student goals) 

(table continues) 
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TABLE 11 (continued) 

ITEM RESPONSES: OTHER TEACHING AND LEARNING MEASURES 
Student activities index 

Satisfaction of employers five years after graduation 
Satisfaction of students with the learning process [3] 
Note . Measures have been edited for clarity as required. 
[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 

sub-processes is located in Appendix A. 
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TABLE 12 

ITEM RESPONSES: OTHER ENROLLMENT MANAGEMENT MEASURES 
Amount of student admission expense 

Amount of student academic success (subjective) [2] 

Benchmarks of best-in-class processes 

Cycle time of application/inquiry processes [2] 

Diversity data of faculty, staff, administration, and 
students (goal dependent) 

Percentage of in-state/out-of-state/international students 
Rate of graduate placement 

Ratio of achieved- to- targeted admissions [3] 

Ratio of achieved- to- targeted enrollments 
Ratio of scholarship-to-nonscholarship enrollments 
Ratio of scholarship endowments- to-recipients 
Ratio of scholarship recipients-to-matriculants 
Ratio of males-to-f emales 

Satisfaction of faculty/students with enrollment process [4] 
Statistical data of Student Achievement Test (SAT) [2] 

Survey of student class scheduling preference (2.7) 

Survey of transf erred/accepted but not enrolled students 
Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 

sub-processes is located in Appendix A. 
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TABLE 13 

ITEM RESPONSES: OTHER ENHANCEMENT OF THE CULTURAL 
ENVIRONMENT MEASURES 

Amount of budget allocated 

Amount of receptiveness to other cultures 

Diversity data of event attendees 

Number of business partnerships 

Number of countries visited or represented 

Number of english as a second language/minority students 

Number of racial conflicts 

Number of required off site-experiences 

Percentage of international students 

Percentage of students traveling abroad 

Quality of events and evaluations [2] 

Rate of student co-curricular participation 
Survey of climate 

Survey of students using focus groups 
Type/variety/diversity of events [3] 

Note . Measures have been edited for clarity as required. 
[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 
sub-processes is located in Appendix A. 
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TABLE 14 

ITEM RESPONSES: OTHER STRATEGIC 
PLANNING/DEVELOPMENT/BUDGETING/MEASURES 

Amount of campus involvement in accomplishing goals 
Quality of institutional alignment in support of goals 
Quality of formal annual built-in reviews 
Rate of participation in annual fund 
Survey of faculty/staff /administration 
Strategic plan/university wide/department level 
Satisfaction of stakeholders (legislators/businesses) [2] 
Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 

sub-processes is located in Appendix A. 

TABLE 15 

ITEM RESPONSES: OTHER RESOURCE 
ACQUI S I T I ON/DEVELOPMENT /MANAGEMENT / ACCOUNTAB IL I TY MEASURES 

Amount of advancement budget /student or revenue 

Amount of increase in tuition/fee costs (trend analysis) 

Amount of budget deficits 

Amount of alumni support revenue 

Amount of salary equity 

Amount of increase in overtime (trend analysis) 

Amount of financial fundraising capability (study) 

(table continues) 
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TABLE 15 (continued) 

ITEM RESPONSES: OTHER RESOURCE 
ACQUIS ITION/DEVELOPMENT/MANAGEMENT/ACCOUNTABILITY MEASURES 

Cost of deferred maintenance [2] 

Cost per full time equivalent ( FTE ) 

National Association of College and University Business 
Officers, administrative benchmarks. 

Number of learning opportunities for staff 
Percentage of budget in payroll benefits 
Quality of faculty/staff (salary comparisons) 

Quality of management letter/audit 

Quality (soundness) of performance appraisal system 
Rate of faculty/staff turnover [2] 

Ratio of facility square footage- to-maintenance costs 

Ratio of faculty- to- staff 

Ratio of requested-to-granted funding 

Ratio of tuition dollars-to-non-tuition sources 

Ratio of students-to-fees 

Ratio of full time-to-part time students 
Ratio of instructional-to-administration accounts 
Satisfaction of faculty/staff /alumni using an annual survey 
Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 

sub-processes is located in Appendix A. 
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TABLE 16 



ITEM RESPONSES: OTHER INSTITUTIONAL MANAGEMENT/GOVERNANCE 

MEASURES 

Assessment of organizational climate (shared vision) [2] 
Content analysis of issues addressed and decisions made 
(minutes of meetings could be analyzed) 

External benchmarking 
Number of cutting edge programs 
Quality of institutional reputation 
Quality of centers for excellence 
Satisfaction of students 

Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 
sub-processes is located in Appendix A. 
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TABLE 17 

ITEM RESPONSES: OTHER RESEARCH/SCHOLARSHIP AND CREATIVE 

ACTIVITY MEASURES 

Amount of faculty output (all types) 

Amount of information requested regarding projects 
Amount of undergraduate student research 
Amount of goal achievement 

Amount of professional recognition earned by f aculty/staf f 
(e.g. fellow status in learned society) 

Amount of new program development/support [2] 

Amount of knowledge (created/applied) impact 

Amount of scholarship impact 

Number of students supported 

Number of student awards 

Number of grants (by department) 

Number of consulting activities for business/government [2] 
Percentage of academic program budget used for research [2] 
Percentage of faculty receiving monetary awards 
Percentage of faculty in public presentations/performances 
Percentage of student involved in research 
Publications per dollar 

Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 
sub-processes is located in Appendix A. 
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TABLE 18 

ITEM RESPONSE: OTHER SERVICE (COMMUNITY OUTREACH) MEASURES 



O 
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Assessment of needs 

Amount of contributions by local government and business 

Amount of applied knowledge impact 

Amount of internal service within institution 

Amount of economic impact 

Amount of service impact on community 

Amount of budgeting/staffing to support outreach 

Diversity of institutions and languages 

Number of activities 

Number of country and foreign institution partnerships [2] 
Number of staff /faculty/student participants [3] 

Number of community service hours 

Number of joint ventures with businesses 

Number of new initiatives/programs generated [2] 

Rate of graduation/retention 

Quality of community perception of the institution 

Rate of participation 

Satisfaction of stakeholders 

Study of image 

Survey of community/region 

Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 

sub-processes is located in Appendix A. 
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TABLE 19 

ITEM RESPONSES: OTHER DEVELOPING/MAINTAINING LEARNING 

ENVIRONMENT MEASURES 

Amount of student usage and participation 
Number of interviews per student 
Number of faculty publications in learning 
Peer institutions comparisons [2] 

Percentage of students in research/presentations 
Peer composition re: electronic classrooms 
Rate of student placement [4] 

Retention data 

Satisfaction of community/users/students/employers [2] 

Survey of climate 

Quality of general and athletic student academic performance 
Number of social problems 

Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 
sub-processes is located in Appendix A. 
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TABLE 2 0 

ITEM RESPONSES: OTHER COMMUNICATING THE INSTITUTIONAL IMAGE 

MEASURES 

Amount of community support 

Amount of national recognition published in external media. 

Amount of marketing budget 

Amount of student /faculty recruitment 

Number of press release appearances/feature articles 
Number of awards/honors in national competitions 
Quality of student perception 
Quality of institutional image 

Quality of publications using content analysis 
Research studies of market as required 

Ratio of positive-to-negative local/external media coverage 
Surveys of public/students/image/marketing/customs 
Satisfaction of faculty/staff /stakeholders/alumni [6] 
Satisfaction of advisory boards using focus groups 
Note . Measures have been edited for clarity as required. 

[ ] = number of respondents proposing measure; 

( ) = sub-process identification number. A listing of 
sub-processes is located in Appendix A. 
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TABLE 21 

ITEM RESPONSES: PROPOSED KEY PROCESSES AND MEASURES 
Key Process 

Faculty & staff leadership development 
Measures 

Participation in growth activities 
Ratio of sought-to-attained leadership positions 
Key Process 
Placement 
Measures 

Satisfaction of alumni 

Percentage of students using services 
Percentage of students attending graduate schools 
Key Process 

Student success 
Measures 

Percentage of students graduating in 4 or more years 
Rate of graduate school acceptance 
Job placement data 
Key Process 

Innovative program developed 
Measures 

Number of new initiators in learning environment 
reputation building 

Percentage of alumni who refer family and friends 
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(table continues) 
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TABLE 21 (continued) 

ITEM RESPONSES : PROPOSED KEY PROCESSES AND MEASURES 
Key Process 

Promoting teaching and curriculum development as an 

alternative to research in esoteric areas for proof 
of scholarly achievement 
Measures 

Number of publications in learning fields 
Number of full professors promoted based on 
scholarship in earning promotion 

Key Process 

Promotion of interdisciplinary team work 
Measure 

Number of cross-college team-taught courses 
Number of cross-college senior projects 
Number of industry and business sponsored 
interdisciplinary team projects 

Key Process 

Promotion of the university as a learning organization 
Measures 

Number of people rewarded for innovative process 
improvement 

Number of workshops/training events sponsored for 
faculty and staff 

Number of training hours per faculty and staff members 





(table continues) 
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TABLE 21 (continued) 

ITEM RESPONSES: PROPOSED KEY PROCESSES AND MEASURES 
Key Process 

Institutional research 
Measures 

Integrated assessment plan status 
Staffing and budget to support assessment 
Key Process 

Financial performance 
Measures 

Fund balance growth 
Key Process 

Quality of residential experience 
Measure 

Student surveys 
Key Process 

Quality of dining services (other support services) 
Measures 

Student surveys (relates to teaching and learning) 
Student satisfaction with teaching, advising, general 
education courses, major courses 

Key Process 

Student selection 

Academic and administrative staff selection 
Academic and administrative staff development 

(table continues) 
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TABLE 21 (continued) 

ITEM RESPONSES: PROPOSED KEY PROCESSES AND MEASURES 
Key Process 

Faculty and staff satisfaction 
Measure 

Organizational practices assessment 

Quality administrators also provided the following 
comments in response to the second survey research question. 

1. You may wish to refer to the NACUBO benchmarking project. 
No need to reinvent or repeat what colleges and 
universities have already agreed to nationally about key 
processes and measures . 

2. Rather than peer institutions, try to identify one or two 
best-in-class for particular process. 

3. Be aware of different missions for different types of 
institutions according to two-year and four-year Carnegie 
Classifications . 

In response to this comment the author has provided 
the Carnegie Classification Codes of the study 
participants in the Directory of Respondents (Appendix D) . 





CHAPTER 5 



CONCLUSION 
The Findings 

The purpose of this study was to (1) identify; and (2) 
verify key processes and associated measures that can help 
higher education quality administrators implement a 
continuous improvement program and subsequent benchmarking 
process for their respective institutions. The study 
findings indicated that an 80% or greater agreement rate was 
obtained for 8 of the 10 key processes and for 16 of the 34 
associated measures. Based on these results the author 
concludes that those key processes and measures obtaining an 
80% or greater agreement rate can serve institutions 
requiring a framework for implementing benchmarking for 
continuous improvement in higher education. 

Teaching and learning and enrollment management 
obtained the highest agreement rate (88%) as a key 
processes. Student retention data (84%) and exit 
examinations (88%) were identified as appropriate measures 
for benchmarking teaching and learning. Retention data 
(92%) and student enrollment (88%) and student satisfaction 
(88%) were cited as appropriate measures for benchmarking 
enrollment management. 
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It was difficult to find absolute measures for key- 
processes that were creative and holistic in nature. For 
example, 80% agreed that enhancement of the cultural 
environment was a key process. However, they did not find 
the measures of number of events (68%) or peer institutions 
comparisons (56%) to be comprehensive. The same findings 
were true for research, scholarship, and creative activity 
along with service and community outreach. Although 80% and 
84% respectfully agreed that these are key processes, 
absolute measures (except for peer institution comparisons 
at 84%) were difficult to find. This suggested an important 
area for further study. Perhaps performance indicators, 
rather than absolute measures, should be sought for the 
creative processes that are fundamental to success in higher 
education . 

Within the context of benchmarking for continuous 
improvement in higher education, this study has identified 
eight key processes and sixteen measures that should be in 
any framework for implementation. The areas for further 
refinement are consistent with these findings while helping 
to facilitate continuous improvement without overly 
restricting the creative dimension of higher education. 

The Recommendations 

Based upon the findings described above the author 
recommends performing an additional study. The proposed 
objective of this study would be to (1) conduct a survey of 
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quality administrators to collect both performance and 
progress indicators of key processes; and (2) conduct a 
second survey to achieve quality administrator consensus of 
the proposed indicators. Tables 11-21 contained in this 
study can serve as a basis for this research. 

It is also recommended that a dissertation format be 
used to conduct a more extensive higher education 
benchmarking study. The objective of this study would be to 
(1) collect actual benchmarking data using the East 
Tennessee State University Key Processes and Measures; (2) 
compare this data with another institution that has 
collected similar data; and (3) identify problems resulting 
from the exchange and prescribe solutions for improvement. 
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APPENDIX A 



ETSU MATRIX OF KEY PROCESS RELATIONSHIPS 
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GLOSSARY OF TERMS: 



AA 

AD 

AREM 

AVP 

BF 

CAPS 

CASS 

COM 

CSPS 

DGS 

DO 

EEO/AAO 

FAE 

FAP 

FAR 

HA 

HR 

ICGME 

IEP 

IR 

MEAC 

MSEC 

PFAA 

SA 

SAC 

SAI 

TLC 

UA 

UR 

VP 

VPs 



Academic Affairs 
Assistant Dean 

Admissions, Retention, & Enrollment Management 
Associate Vice President 
Business & Finance 

Center for Adult Programs & Services 
Center for Appalachian Studies & Services 
College of Medicine 

Continuing Studies & Public Service 
Dean of the School of Graduate Studies 
Director of 

Equal Employment Opportunity/Affirmative Action 
Officer 

Faculty Activity Evaluation 
Faculty Activity Plan 
Faculty Activity Report 
Health Affairs 
Human Resources 

Institution Committee on Graduate Medical 
Education 

Institutional Effectiveness & Planning 
Information Resources 

Medical Education Assistance Corporation 

Medical Student Education Committee 

Performance Funding & Academic Assessment 

Student Affairs 

Student Activities Center 

Student Assessment of Instruction 

Teaching & Learning Center 

University Advancement 

University Relations 

Vice President 

All Vice Presidents 



KEY SUCCESS FACTORS: 



1 Educate students to become responsible, enlightened, & 
productive citizens. 

2 Conduct scholarship that improves the human condition. 

3 Serve business, education, government, health care 
systems, community. 

4 Enhance the cultural environment of the region. 

5 Improve stewardship and institutional effectiveness. 

6 Improve resource acquisition (human & financial) . 
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January 29, 1996 

Dr. Judith A. Gaston 
University of Minnesota Quality 
1313 5th St., S.E. Ste. 108 
Minneapolis, MN 55414 

Dear Dr. Gaston: 

We respectfully request your assistance with a 
graduate research study. In appreciation for your efforts, 
a copy of the study results will be provided if you so 
indicate by returning the enclosed postcard. The primary 
study objective is to reach a consensus among higher 
education quality administrators on key "institutional" 
processes and performance measures that would allow for 
focused data collection and comparison for future 
benchmarking investigations between institutions. 

The attached survey instrument is a short 
questionnaire that outlines ten key institutional processes 
along with thirty- four corresponding performance measures as 
developed by the East Tennessee State University Strategic 
Planning Committee. Related sub-processes are also included 
to aid in understanding the scope of the key processes. To 
respond simply circle agree or disagree and list any "other" 
key processes or measures you feel are appropriate in the 
space provided. Be assured that your individual responses 
are completely anonymous. 

To ensure successful completion of this study your 
responses need to be returned via enclosed envelope before 
February 19th, 1996. If you have any questions we can be 
reached by phoning (423) 929-4465; (423) 282-4124 or Faxing 

(423) 929-5743. 

We greatly appreciate your efforts in helping to 
accomplish the objectives of this study. 

Sincerely, 



Robert G. Stewart 
Thesis Author 



James A. Hales 

Dean College of Applied Science and Technology 
Chair University Strategic Planning Committee 
Thesis Advisor 
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INSTITUTIONS -KEY INFORMANTS -CARNEGIE CLASSIFICATION CODES 



Auburn University 

Dr. Bettye B. Burkhalter 


Tel : 


(334) 


844-2000 


Assistant Provost 


Fax : 


(334) 


844-2001 


Assessment Sc Quality Improvement 


E 


-mail : 


burkhbb@ 


209 Samford Hall 




mail . auburn . edu 


Auburn, AL 36849 
ccc : R-I 








Belmont University 

Dr. Susan G. Hillenmeyer 


Tel : 


(615) 


385-6441 


Vice President 


Fax : 


(615) 


386-4555 


Quality Sc Professional Development 


E-mail 


: hillenmeyers@ 


1900 Belmont Blvd. 




belmont . edu 


Nashville, TN 37212 
ccc: R-II 








Cleary College, Ypsilanti 

Dr. Tom Sullivan 


Tel : 


(313) 


483-4400 


President Sc Chief Executive Officer 


Fax: (313) 483- 


0090 


2170 Washtenaw Ave . 
Ypsilanti, MI 48197 
ccc: R-I 








David Lipscomb University 

Dr. Richard W. Kulp 


Tel : 


(615) 


269-1833 


Assistant Vice President 


Fax : 


(615) 


269-1818 


Quality Sc Institutional Efficiency 




E-mail 


: kulprw@ 


P.O. Box 4236 
Nashville, TN 37204-3951 
ccc: R-I 






dlu . edu 


Duke University 

Dr. Jane Tucker 


Tel : 


(919) 


613-7608 


Director 


Fax : 


(919) 


613-7620 


Training Sc Organizational 


E- 


mail : 


tucker j a@ 


Development 


mailol . adm 


. duke . edu 


402 Oregon St. 
Durham, NC 27705 
ccc: D-I 








Georgia College 

Dr. Robert E. Craft Jr. 


Tel : 


(912) 


471-2990 


Quality Improvement Coordinator 
Macon Campus 

3920 Arkwright Rd. , Ste. 160 
Macon, GA 31210-1719 
CCC: D- II 


Fax : 


(912) 


471-2985 
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INST I TUT IONS -KEY INFORMANTS -CARNEGIE CLASSIFICATION CODES 



Georgia Institute of Technology 

Dr. Hal Irvin 


Tel : 


(404) 


894-1099 


Director 


Fax : 


(404) 


853-9163 


Continuous Quality 


E-mail: hal.irvin@ 


Improvement 


success . gatech . edu 


225 North Ave . , NW Carnegie Bldg. 
Atlanta, GA 30332-0325 
ccc: R-I 








Huntington College 

Dr. Robert K. Boozer 


Tel : 


(219) 


356-6000 


Assistant to the President 


Fax : 


(219) 


356-9448 


2303 College Ave. 


E 


-mail : 


rboozer@ 


Huntington, IN 46750 




huntcol . edu 


ccc: C-I 








Illinois, Chicago (University of) 

Dr. Robert S. Winter 


Tel : 


(312) 


413-3600 


Associate Chancellor 


Fax : 


(312) 


413-3606 


Quality Advancement 


E-mail: robert . 


s . winter@ 


715 S. Wood, Ste. 301 ASB (M/C 104) 
Chicago, IL 60612 
ccc: C-II 






uic . edu 


Inter American University of Puerto Rico 

Dr. Ileana Iruinse 


Tel : 


(809) 


763-4633 


System Quality Coordinator 
P.0. Box 363255 
San Juan, PR 00936 
ccc: M-I 


Fax : 


(809) 


751-2190 


Michigan State University 

Dr. Robert F. Banks 


Tel : 


(517) 


353-5300 


Assistant Provost & Vice President 


Fax : 


(517) 


355-9601 


Academic Human Resources 


E 


-mail : 


rfbanks@ 


422 Administration Bldg. 
East Lansing, MI 48824-1046 
CCC: M-I I 






msu . edu 


Monterry Institute of Technology 

Dr. Daniel Meade 


Tel : 


(528) 


358-2000 


TQM Coordinator, QINTEC 


Fax : 


(528) 


358-4555 


Sueur sal de Correos "J" 




E-mail 


: dmeade@ 


Monterrey, N.L. 64849 


campus . mty 


. itesm.mx 


Mexico 








ccc : none 
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INST I TUT IONS -KEY INFORMANTS -CARNEGIE CLASSIFICATION CODES 



North Carolina, Charlotte (University of) 

Dr. J. William Shelnutt 

Assoc. Prof, of Engineering Technology 
University Chair 

Quality Improvement Steering Committee 
Precision Engineering 
Charlotte, NC 28223 
ccc: D-II 


Tel : 
Fax : 
E- 


(704) 

(704) 

mail: 


547-4370 
547-3246 
shelnutt@ 
uncc . edu 


North Carolina State University 

Dr. Karen Helm 
Assistant Provost 
P.O. Box 7101 
Raleigh, NC 27695-7101 
ccc: D-II 


Tel : 
Fax : 
E 


(919) 
(919) 
-mail : 
gwgate 


515-7528 
515-1686 
nprvrel@ 
. ncsu . edu 


Northeast Missouri State University 

Dr. Ralph P. Cupel li 

Assistant to the Vice President 

Academic Affairs 

McClain Hall 203 

Kirksville, MO 63501 

ccc: M-II 


Tel: (816) 785-4221 

Fax: (816) 785-7460 

E-mail: aeo3%nemomus@ 
academic . nemdstate . edu 


Oregon State University 

Dr . Nancy Howard 
Director 

Quality & Continuous Improvement 
Administrative Services A638 
Corvallis, OR 97331-2128 
ccc: M-I 


Tel : 
Fax : 
E 


(503) 
(503) 
l-mail : 
ccmail 


737-0548 
737-3033 
howardn@ 
. orst . edu 


Portland State University 

Dr. A. J. Arriola 

Coordinator for Quality Initiatives 

P.O. Box 751 

Portland, OR 97207-0751 

CCC: M-I 


Tel : 
Fax : 


(503) 

(503) 


725-4788 

725-5800 


Rutgers University, New Brunswick 

Dr. Brent D. Ruben 
Executive Director 


Tel: 
Fax : 


(908) 

(908) 


932-1420 

932-1422 



Quality & Communication Improvement E-mail: ruben@ 

4 Huntington St. qci.rutgers.edu 

New Brunswick, NJ 08903 
ccc : R- I 
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INSTITUTIONS -KEY INFORMANTS -CARNEGIE CLASSIFICATION CODES 



Samford University 

Dr. John Harris 
Assistant to the Provost 
Quality Assessment 
Birmingham, AL 35229 
ccc : D-I 




Tel: (205) 870-2674 

Fax: (205) 870-2908 

E-mail: jwharris@ 
mailbox . samford . edu 


San Juan College 

Dr. James Henderson 
President 

4601 College Blvd. 
Farmington, NM 87402 
ccc: D- II 




Tel: (505) 599-0228 

Fax: (505) 599-0385 


Stevens Institute of Technology 

Dr. Joseph Moeller Jr. 

Vice President & Director 
Quality Process 
Castle Point on Hudson 
Hoboken, NJ 07030 
ccc: D-II 


Tel : 
Fax : 


(201) 216-5229/216-5228 

(201) 216-8044/216-8326 

E-mail: jmoeller@ 
stevens-tech. edu 


Texas Tech University 

Ms. Kerry Billingsley 
Quality Service Department 
Box 2015 

Lubbock, TX 79409-2015 
CCC: M-I 




Tel: (806) 742-0530 

Fax: (806) 742-2241 

E-mail: qualt@ 
ttacs . ttu . edu 


Union College 

Ms. Linda Wysing 

Quality Improvement Coordinator 
3800 S. 48th St. 

Lincoln, NE 68506 
ccc: R-I 




Tel: (402) 486-2540 

Fax: (402) 486-2895 

E-mail: liwysing@ 
snoopy . ucollege . edu 


Villanova University 

Dr. John M. Kelley 
Director 

Planning & Institutional Research 
800 Lancaster Ave . 

Villanova, PA 19085-1699 
ccc: R-III 




Tel: (610) 519-4835 

Fax: (610) 519-7162 
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Weber State University 

Dr. Cherrie Nelson 


Tel : 


(801) 


626-7496 


Quality Director 


Fax : 


(801) 


626-7922 


Administrative Services 


E 


-mail : 


cnelson@ 


3750 Harrison Blvd. 
Ogden, UT 84408-1006 
ccc: RU-I 




cc . 


weber . edu 


Western Michigan University 

Dr. Barbara S. Liggett 


Tel : 


(616) 


387-3655 


Assistant Vice President 


Fax : 


(616) 


387-2355 


Executive Advisor 


E-mail: barbara 


. liggett@ 


for Quality 

3070 Seibert Administration Bldg. 
Kalamazoo, MI 49008 
ccc: CU-I 






wmu . edu 


University of Wisconsin, Madison 

Dr. Maury Cotter 
Director 

Quality Improvement 
RM 199 Bascom Hall 
500 Lincoln Drive 
Madison, WI 53706-1380 
ccc: CU-I I 


Tel : 


(608) 


262-9313 



Note : All information was current as of 18 May 1996. 
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